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PREFACE 


This  report  was  sponsored  by  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  and 
monitored  by  the  U.S.  Army  Topographic  Engineering  Center  (TEC)  under  contract  DAC76-92-C- 
0024,  titled,  "Site  Model  Based  Image  Registration  and  Change  Detection."  The  DARPA  Program 
Manager  was  Dr.  Thomas  Strat,  and  the  TEC  Contracting  Officer's  Representative  was  Ms.  Lauretta 
Williams. 


1  Introduction 


The  University  of  Maryland  (UMD)  is  contributing  model-based  and  context-based  change 
detection  (CD)  and  monitoring  algorithms  to  the  RADIUS  project.  Change  detection 
(CD)  and  monitoring  are  essential  aerial  image  exploitation  tasks  [2].  Previous  research  in 
this  area  has  relied  on  general-purpose  methods  that  are  unreliable  because  of  changes  in 
weather,  illumination,  and  imaging  conditions.  The  model-based  detection  schemes  pre¬ 
sented  here  incorporate  IU  techniques  whose  primitives  can  be  used  to  direct  the  system  to 
conduct  spatially  constrained  analyses,  whose  outcomes  may  be  indicative  of  occurrences 
of  changes  with  intelligence  significance.  The  aerial  image  exploitation  system  described 
here  is  site-model  driven  and  emphasizes  using  geometric  (i.e.  object  and  spatial)  primi¬ 
tives,  which  appear  as  site  models  implemented  in  the  RADIUS  RCDE  environment  [l]. 
The  models  encode  the  spatial  relationships  between  fixed  objects  of  interest  in  a  site,  such 
as  buildings,  roads,  etc. 

In  this  report,  progress  in  context-based  aerial  image  analysis  is  summarized.  Model- 
based  approaches  are  presented  for  the  following:  (a)  detecting  and  counting  vehicles  on 
parking  lots  as  well  as  a  long  road;  (b)  monitoring  object  formations,  such  as  convoys  on 
roads  or  railroad  cars  on  tracks.  In  our  system,  specific  approaches  have  been  matched  to 
the  tasks  at  hand.  A  global  method  using  spectral  analysis  is  used  for  (b),  and  template 
matching  is  used  for  (a).  Site  model  information  is  then  incorporated  in  different  ways  and 
to  various  extents,  according  to  the  selected  detection  strategy. 

Section  2  reports  on  detection  of  convoys  and  other  configurations  and  Section  3 
presents  improvements  to  the  vehicle  detection  and  counting  module,  with  special  em¬ 
phasis  on  integration  into  the  RADIUS  testbed.  In  each  section,  experimental  results  are 
given. 

2  Monitoring  Vehicle  Formations 

We  considered  specific  site-model-supported  monitoring  tasks  involving  movable  objects 
and  vehicles.  In  the  first  task,  vehicles  in  selected  regions,  such  as  in  parking  areas,  roads, 
and  training  grounds,  were  detected  and  counted.  In  the  second  task,  vehicle  formations, 
such  as  convoys,  were  detected. 

In  this  section,  a  detector  is  described  that  may  be  used  as  a  tool  for  globally  qualifying 
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the  degree  to  which  a  parking  lot  is  full  or  determining  the  presence  of  platoons  or  convoys 
along  roads;  this  detector  is  primarily  intended  as  an  attentional  mechanism.  Our  approach 
is  based  on  detecting  special  vehicle  configurations  -  more  specifically,  configurations  that 
exhibit  a  periodic  behavior.  This  module  is  of  particular  interest  since  it  enables  one  to  look 
globally  for  activity  in  an  entire  area,  a  process  referred  to  as  Global  Activity  Detection 
(GAD).  It  may  also  serve  as  a  preprocessing  step  to  the  local  vehicle  detection  process. 

2.1  Principle  of  Operation 

Let  r  denote  the  position  of  a  point  in  the  image,  let  C  :  g( r)  =  0  be  the  representation  of 
a  road  center,  let  s  denote  the  curvilinear  abscissa  on  C,  and  let  rc(s)  be  the  image  point 
position  as  a  function  of  s.  Further  denote  by  &(s)  the  directional  derivative  along  the 
tangent  to  C  to  the  curve  averaged  over  L(s)  in  the  normal  direction  at  position  rc(s),  i.e. 

k(s)  =  DtI(rc(s)) 

where  w  is  the  vehicle  width  and  L(s)  =  {r  :  r  =  AVg  +  rc(s),  A  E  (— y,  y)}.  The  presence 
of  organized  convoys  can  be  detected  from  any  regular  or  periodic  behavior  of  directional 
edges.  This  can  be  achieved  from  spectral  analysis  of  k(s).  Let  K(f )  =  F[k\f]  denote 
the  Fourier  Transform  (FT)  of  k(s),  a  procedure  essentially  equivalent  to  using  a  linear 
boost  on  the  FT,  thereby  eliminating  any  DC  component.  A  convenient  model  for  k(s)  for 
regularly  placed  objects,  such  as  a  convoy-like  formation  of  vehicles  on  a  road  or  vehicles 
in  a  sufficiently  full  parking  area,  is  obtained  by  assuming  a  multiply-replicated  function 
c(.)  corresponding  to  the  edge  profile  (the  directional  derivative  profile)  of  a  vehicle,  or 
corresponding  to  a  truncated  periodic  version  of  this  function  with  additive  white  noise 
n(s)  (to  model  clutter  structure),  i.e. 

k(s)  =  rect(^-^)([c(.)  *  |/*|  comb(/*  x  .)]  (s))  +  n  (s)  (1) 

with  rect(x)  =  1  if  |x|  <1/2  and  0  outside,  and  comb(:r)  =  J2n6(x  —  n).  The  resulting 
K(f )  then  equals 

I<(f)  =  smc(.  x  p  *  (C(.)  comb (.//*))]  (/)  +  N(f) 

Detecting  convoys  on  roads  or  detecting  significant  parking  area  occupancy  can  be  carried 
out  by  detecting  impulsive  spectral  components  and  corresponding  harmonics  from  the 
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PSD  (Power  Spectral  Density)  K(f)2.  Various  strategies,  some  of  which  are  described 
later,  can  be  derived  to  measure  this  occurrence. 

The  two-dimensional  case  is  more  interesting  since  it  allows  for  many  possible  object 
configurations.  This  case  is  also  relevant  for  monitoring  parking  area  activity  or  regular 
vehicle  formations  in  open  areas.  Let  k( r)  =  |  V/(r)  |,  the  gradient  magnitude  at  r,  and 
consider  the  2-D  FT, 

K(i)  =  F[A:|f]  =  J  j  dr  k{ r)  exp—  j2wrTi 

with  f  =  [/o,  /i]T.  For  regular  2-D  vehicle  formations,  a  totally  general  configuration  model 
is  obtained  by  assuming  arbitrary  replications  of  an  arbitrarily  rotated  object  (see  Figure 
1).  Its  corresponding  gradient  magnitude  can  be  described  as 

k(r)  =  £  c(R"nr  -  vn)  +  n(r)  (2) 

n 

with  vn  and  Rw„  denoting  an  arbitrarily  chosen  2-D  translation  vector  and  rotation  matrix. 
Since 

F[®(Ar)|f]  =  |  det(A)|-1  F1[a;(r)|Af] 
then  with  k( r)  as  in  Equation  (2)  we  have 

#(f)  =  £exP  (-j2x(R**f)rvw)  F[c(.)|RWnf]  +  N( f) 

n 

This  structure,  however,  is  too  loose  to  yield  any  useful  spectral  signature.  Further  restrict¬ 
ing  k( r),  another  convenient  and  still  general  configuration  model  is  obtained  by  letting 
Ru,n  =  Ra/  in  Equation  (2)  (same  object  orientations)  and  by  assuming  periodic  replication 
of  the  object  within  a  support  area  A.  Directions  of  replication  are  defined  by  the  vectors 
Vo  and  Vi,  or  equivalently  by  the  2-D  periodicity  matrix  V  whose  columns  are  the  v,-’s, 
i  =  0, 1  (see  Figure  1),  with  det(vo,  Vi)  =  det(V)  ^  0.  This  situation  is  general  enough  to 
describe,  by  simple  selection  of  the  shape  of  the  support  region  A,  a  wide  array  of  config¬ 
urations  of  special  interest  such  as  circular,  semi-circular,  wedge,  pyramid,  block  or  linear 
formations,  a  few  examples  of  which  are  shown  in  Figure  2.  Then,  letting  Ta( r)  denote 
the  indicator  function  over  A,  the  resulting  spatial  representation  is  given  by 

H r)  =  T4r)([c(Rw.)  *  *comb2(VOfVl)(.)]  (r))  +  n  (r) 
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with  the  separable  comb2(r)  defined  as  comb2(VoVl)(r)  =  Sn^(r  +  ^n))  with  n  =  [i,j]T- 
Its  resulting  spectral  representation  is 


K{ f)  =  [IA(.)  *  *(C(Ro,.)  Comb2(V0)Vl )(.))]  (f)  +  N( f) 


where  Ia(-)  =  jP[X^4 jf] ,  and  Comb2(v0,Vl)(f)  satisfies 

Comb2(VOpv,)(f)  = 


As  in  the  1-D  case,  a  particular  configuration  can  be  detected  by  searching  for  its  corre¬ 
sponding  2-D  base  and  harmonic  impulsive  spectral  domain  components. 

Constraints,  which  are  provided  by  a  3-D  geometrical  model  of  the  vehicle,  yield  spec¬ 
tral  compliance  windows  where  one  can  search  for  impulsive  components  characteristic  of 
certain  types  of  regular  configurations.  Let  /*  denote  the  base  component  arising  as  a 
result  of  a  regular  vehicle  formation  on  a  road,  and  let  No  be  the  sampling  rate  along 
the  curve  C.  The  average  image  vehicle  length  /  taken  on  the  image  region  covered  by 
the  road  is  derived  from  the  known  vehicle  model  and  the  camera  orientation  parameters. 
For  safety  reasons,  vehicles  follow  each  other  at  distances  greater  than  el,  for  some  e  >  0; 
an  upper  bound  for  /*  is  then  given  by  No/((l  +  e)l).  Similarly,  vehicles  following  each 
other  at  great  distances  are  not  considered  organized  convoys.  Alternatively,  if  the  area 
under  study  is  a  parking  area,  it  is  considered  as  a  significantly  occupied  parking  area. 
The  maximum  distance  can  be  expressed  in  terms  of  vehicle  lengths  l  as  pi  with  p  >  e. 
This  yields  a  lower  bound  ATo/((l  +  p)l)  for  the  dominant  spectral  component.  In  sum,  /* 
must  lie  within  a  spectral  compliance  window  Wc ,  defined  as 


/*  e  wc  with  wc  = 


/: 


No 


(1  +  p)l 


</< 


No  ] 
(1  +  e)l  J 


A  convoy  is  hypothesized  if  an  impulsive  component  is  found  at  frequency  /*,  also  referred 
to  here  as  the  base  or  dominant  spectral  component.  Then  f*  must  satisfy 


/*  =  argmax(|A'(/)|),  where  /  G  Wc 

When  the  dominant  impulsive  component  is  found  in  the  spectral  compliance  window, 
corroborating  evidence  is  subsequently  checked  at  harmonic  values  of  /*.  Detection  is 
thereafter  declared  if  a  set  of  conditions  are  met  for  the  corresponding  power  at  f*  and 
inside  the  spectral  compliance  window  (as  well  as  on  the  corresponding  harmonics).  These 
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conditions  are  reported  later  in  this  section.  Similar  compliance  windows  are  derived  in 
the  two-dimensional  case.  Consider  the  periodicity  matrix  decomposition  as 

V  =  V  diag(r0,  Ti),  V  =  [v0|vi] ,  |v0|  =  |vx|  =  1 

The  spectral  periodicity  matrix  U  =  V_T,  so  that  U  =  V_rdiag(r0-1,  rf1).  Also,  u,.Vj  = 
=  0,1,  since  UTV  =  (V  r)TV  =  I.  Therefore,  the  spectral  periodicity  vectors 
Uj  are  orthogonal  to  their  alternate  spatial  counterparts,  i.e  u,.Vj  =  0  if  i  ^  j,  Also,  from 
u,.v,  =  1,  and  v»  =  t,-v;,  we  have  |u,|  =  |  cos(u,-,  v,)!-1^-1  =  |  sin(vj,  with  j  —  i, 

and  we  have  the  decomposition 

u;  =  u,-&,with& 

where  it  is  assumed  that  det  (v0,vi)  ^  0. 
write  the  gradient  magnitude  map  as 

A;(r)  =  ^  c(.)  *  £(r  —  nv) 

n 

for  which 

*W  =  c(.)E«(vTf-0 

l 

which  gives  rise  to  impulsive  lines  orthogonal  to  the  direction  of  v.  In  sum,  the  spectral 
signature  offers  clues  to  the  spatial  geometry  of  the  detected  configuration,  and  we  may 
therefore  determine  spatial  orientations  of  the  object  configurations.  Alternatively,  if  site 
model  information  is  present,  compliance  windows  can  be  derived  to  guide  the  detection 
process. 

Using  geometrical  considerations  similar  to  the  ones  presented  in  the  one-dimensional 
case,  the  following  compliance  (longitudinal  and  lateral)  constraints  may  be  inferred: 


=  ^  1  |sin(v0,vi)|  1  (3) 

Otherwise,  in  the  degenerate  case,  one  may 


No 


(1  +  pw)w 
No 


<  fru0|sin((v0,vi)|  < 


No 


(1  +  fi>i)l 


<  fJUi  |  sin((v0,Vi)|  < 


(1  +  ew)w 
No 


(1  +  ti)l 


(4) 


The  search  strategy  to  detect  a  compliant  impulsive  component  follows  various  schemes 
depending  on  the  available  model  information.  If  only  the  direction  of  the  vehicle  is 


5 


available,  then  the  constraints  in  Equation  (4)  translate  into  radial  search  spaces  on  a  half¬ 
plane  along  directions  Uo  and  Ui,  or  alternatively  into  a  search  sector  if  an  uncertainty 
on  these  directions  is  specified.  If  only  the  object  dimensions  are  known,  the  search  is 
conducted  along  annular  disks,  generated  by  a  full  2t  revolution  of  the  sets  in  Equation  (4). 
Finally,  if  both  dimension  and  directions  can  be  inferred  from  the  site  model  or  from  the 
context,  the  search  space  reduces  to  annular  sectors. 

We  turn  next  to  the  detection  strategy.  A  significant  component  due  to  clutter  noise  is 
present  across  the  spectral  compliance  window,  and  detection  is  often  complicated  by  poor 
imaging  and  illumination  conditions,  which  lead  to  weak  gradient  magnitude.  Inaccuracies 
in  site  and  road  boundary  models,  as  well  as  occlusions  and  shadows,  may  lead  to  false 
alarms.  With  these  complications  in  mind,  the  following  detection  rule  is  chosen  for  an 
impulsive  component  at  a  base  and  corresponding  harmonic  frequencies: 

Detection  is  declared  if  conditions  (a)  and  (b)  are  met  simultaneously: 

1.  Power  Ratio  Test :  \Kr(f*)\  >  /c, 

2.  Absolute  Power  Test:  |A'(/*)|  >  7 

where  Kr(f)  =  K(f)/Kmed,  and  Kme<i  denotes  the  “median”  power  inside  the  spectral 
compliance  window.  If  these  conditions  are  met,  then  the  corresponding  harmonics  of  f* 
are  checked  for  the  presence  of  an  impulsive  component  using  the  above  decision  rules,  i.e. 

>  Ki  and  \K(f*)\  >  7 where  the  subscript  i  indicates  the  ith  harmonic  of  the 
base  frequency  /*,  /;*  =  if*.  In  practice,  it  is  often  difficult  to  verify  more  than  the  first 
harmonic.  Furthermore,  in  order  to  simplify  tuning  and  keep  thresholds  constant  across 
images  and  viewing  directions,  the  spectral  estimates  are  normalized  by  the  aspect  ratio. 
Call  the  resulting  normalized  power  the  absolute  power  Ka.  Further  tests  can  involve 
comparisons  with  the  spectral  components  of  k(.)  for  other  images  where  the  given  road 
is  known  to  be  empty. 

Varying  illumination  conditions  lead  to  variations  in  power  associated  with  a  given 
directional  derivative  map.  To  allow  for  these  variations,  the  above  detection  rule  on  the 
absolute  power  Ka  and  power  ratio  I\r  is  replaced  by  an  acceptance  region  defined  on  the 
space  (Ka,Kr).  This  acceptance  region  is  based  upon  the  premise  that  peaks  with  lower 
absolute  power  should  be  tolerated  for  larger  power  ratios.  We  define  a  boundary  region 
in  the  space  (Ka,Kr),  Kr  =  b{Ka).  The  detection  rule  then  becomes 

d(Ka,Kr)=lRb(Ka,Kr) 
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where  Tr  is  the  indicator  function  over  region  R,  and  Rb  is  defined  as 

Rh  =  {(Ka,Kr),s.t.I<r  >  b(Ka)} 

Tuning  the  function  b(.)  is  neither  an  acceptable  nor  a  practical  option  due  to  the  large  and 
diverse  amount  of  remotely  sensed  imagery  typically  available  for  exploitation.  The  optimal 
acceptance  region  is  instead  learned  from  a  representative  control  set  of  test  images.  Let 
$  denote  this  control  set.  The  acceptance  region  boundary  is  parameterized  by  vector  V, 
i.e.  6y(.)  (an  example  of  a  possible  parametric  function  is  a  decaying  exponential).  For  the 
previous  detection  rule  d  and  control  set  S,  we  are  interested  in  optimizing  the  choice  of 
V  for  an  objective  function  0(d) ,  which  includes  the  probability  of  false  alarm  Pf(d)  and 
probability  of  detection  Pd(d), 

0(d)  =  Cnd(l-Pd(d))  +  CfaPf(d) 

where  the  coefficients  C/a  and  Cnd,  respectively,  denote  the  selected  costs  associated  with 
occurrence  of  a  false  alarm  and  with  non-detection.  Cnd  and  C fa  reflect  the  priorities  or 
the  final  function  of  the  GAD  (preprocessor,  stand-alone).  In  sum,  V*,  the  optimal  choice 
of  the  parameter  V,  is  one  that  minimizes  the  selected  cost  function  computed  over  the 
control  set  S  of  images,  i.e. 

V*  =  Argmin(0(d)),  with  d(I<a,Kr )  =  lRb(v)(I<a,  Kr)  (5) 

where  Rb(v)  =  {(AT0,  Kr),  s.t.  Kr  >  b(Ka,  V)}.  If  the  GAD  is  used  as  preprocessor  to  local 
object  detection,  then  higher  false  alarm  rates  can  be  tolerated. 

Global  activity  detection  can  naturally  serve  as  a  preprocessing  step  to  local  object 
detection  and  vehicle  detection.  We  can  infer  the  spatial  positions  of  edges,  giving  rise 
to  the  periodic  structure  determined  through  frequency  domain  analysis,  referred  to  here 
as  periodic  loci ,  and  thereafter  apply  local  vehicle  detection  schemes  to  corroborate  the 
presence  of  a  vehicle  at  these  positions.  In  the  1-D  case,  k(s)  is  band-pass-filtered  around 
the  base  frequency  f*  and  harmonic  frequencies  (with  impulse  response  denoted  by  h(s)). 
The  locations  of  edges  satisfying  the  periodic  structure  are  then  determined  by  the  set  of 
locations  s  satisfying  the  condition 

\k(.)  *  h(.)  (s)|  >  vkm  (6) 

for  some  v  <  1,  with  km  denoting  the  median  of  the  absolute  value  of  k(s).  Let  C  denote 
the  set  of  periodic  loci  satisfying  Eq.  (6).  A  local  vehicle  detector  can  then  be  applied  to 
a  reduced  spatial  window  centered  at  these  locations. 
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2.2  Experiments 

The  global  activity  detector  was  run  on  the  complete  set  of  40  RADIUS  model  board  MB2 
images.  Detection  and  false  alarm  rates  are  reported  in  Table  1  for  hand-tuned  and  learned- 
decision  rules.  Results  for  tuned  road  activity  are  given  in  Table  1.  These  detection  results 
are  very  satisfactory,  given  that  the  decision  rule  and  acceptance  region  remain  the  same 
across  images.  Detection  results  are  shown  in  the  following  figures  for  various  images. 
Figure  3(a)  shows  a  newly  acquired  image;  GAD  is  applied  to  operator-selected  areas 
(shown  in  yellow).  Figures  3(b)  and  (c)  show  detected  active  parking  areas.  The  resulting 
detected  active  roads  are  highlighted  in  Figure  3(d)  and  Figure  4(a).  Figure  4  (b,  c  and  d) 
shows  the  results  when  GAD  is  applied  to  all  the  monitored  areas  shown  in  Figure  3(a). 
Active  areas  are  shown  in  red,  and  inactive  roads  are  shown  in  green.  Inactive  parking 
lots  are  not  highlighted.  The  resulting  set  of  periodic  loci  L  is  shown  in  Figure  5(a)  and 
again  in  Figure  5(b)  (enlarged).  After  applying  the  vehicle  detector  described  next,  the 
resulting  detected  vehicles  are  shown  in  Figures  5(c)  and  (d). 

3  Integration  and  Improvements  of  the  Detection  and  Counting  Module 

In  this  section,  we  review  briefly  the  basic  principles  of  the  vehicle  detection  scheme. 
Then,  we  report  on  the  improvements  made  during  the  reporting  period  and  on  the  RCDE 
integration  of  this  module. 

Principle  of  Operation.  The  vehicle  detector  can  be  triggered,  or  cued,  by  the  previous 
GAD  or  can  be  used  independently  to  monitor  areas,  such  as  parking  lots,  roads  or  training 
grounds.  The  general  vehicle  detection  scheme  relies  on  contour  matching  using  informa¬ 
tion  derived  from  the  geometric  model.  As  we  are  primarily  concerned  with  high-altitude 
imagery  in  our  implementation,  vehicles  are  modeled  as  3-D  cuboids  with  given  width, 
length  and  height  specifications.  The  implementation  of  our  vehicle  detector  consists  of 
a  prescreener,  an  extractor  and  a  verifier.  The  first  stage  is  replaced  by  the  hypothesized 
vehicle  locations  at  the  periodic  locus  C  if  the  GAD  is  used  as  a  preprocessor,  with  the  last 
two  stages  remaining  the  same.  The  prescreener  is  applied  instead  if  the  vehicle  detector  is 
used  stand-alone.  This  prescreener  relies  on  Hough  Transform  techniques  to  locate  areas 
where  the  centers  of  vehicles  are  likely  to  lie.  The  extractor  performs  template  matching 
and  hypothesized  vehicles  are  subsequently  examined  in  more  detail.  The  verifier  checks 
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for  shadows  to  guarantee  the  correctness  of  the  results.  Throughout  the  detection  proce¬ 
dure,  3-D  object  models  and  site  information  (camera  model,  illuminant,  etc.)  are  used. 
The  above  vehicle  detection  scheme  has  been  fully  integrated  into  the  RCDE  exploitation 
platform.  Results  of  its  application  to  both  parking  lots  and  roads  are  shown  below  on 
real  images. 

Assessing  Operational  Requirements  for  the  Vehicle  Detector.  We  have  tested 
the  vehicle  detector  on  the  Ft.  Hood  images  so  as  to  assess  reasonable  operational  condi¬ 
tions.  These  conditions  will  be  coded  into  Prolog  rules  in  the  SRI  HUB  [2]. 

Examples  of  results  obtained  with  Ft.  Hood  images  are  shown  in  Figure  6  (a-d).  Table  2 
shows  detection  performance  results  on  a  set  of  Ft.  Hood  parking  lots.  Based  on  these 
experiments,  we  conclude  that  the  current  vehicle  detector  modules  should  be  used  with 
caution  when  any  of  the  following  conditions  are  present: 

•  Poor  contrast. 

•  Very  cluttered  environment  or  very  heterogeneous  objects. 

(such  as  some  Ft.  Hood  storage  areas). 

•  Vehicles  of  greatly  varying  orientations  in  a  parking  lot  when  only  one  orientation 
was  specified. 

•  GSD  (Ground  Surface  Dimension)  such  that  the  image  dimension  of  a  vehicle  is  <  6 
or  7  pixels. 

•  Presence  of  water  pools  or  oil  marks  (noted  by  J.  Mundy). 

The  knowledge  of  these  conditions  is  of  great  interest  to  us.  The  conditions  will  be 
coded  into  Prolog  rules  in  a  RCDE  HUB  subsystem,  which  will  control  the  appropriate 
application  of  detection  algorithms,  according  to  the  operational  conditions. 

An  Automatic  Ground  Truth  Verifier.  The  above  statistics  were  obtained  by  in¬ 
spection.  We  have  now  implemented  an  automatic  ground  truthing  tool  to  allow  for  the 
automatic  assessment  of  the  statistical  performance  of  our  vehicle  detector.  GE  has  devel¬ 
oped  and  integrated  a  very  useful  tool  allowing  systematic  evaluation  of  results  obtained 
with  the  vehicle  detector:  the  ground  truth  verifier.  This  is  primarily  a  developer’s  tool 
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allowing  us  to  perform  a  fast  evaluation  of  the  results  when  a  change  in  the  vehicle  detec¬ 
tion  scheme  has  been  made.  The  current  criterion  for  a  correct  detection  is  the  presence 
of  a  detected  vehicle  within  a  pre-specified  distance  from  the  center  of  a  specified  ground 
truth  vehicle.  This  distance  could  be  replaced  by  a  measure  reflecting  the  GSD  and  the 
vehicle  size. 

We  have  modified  the  ground  truth  verifier  to  allow  for  this.  As  a  first  approximation, 
the  original  detection  criterion  was  modified  to  declare  a  correct  detection  whenever  the 
center  of  the  detected  vehicle  is  within  the  boundaries  of  a  specified  ground  truth  box. 
This  rule  will  later  be  refined  so  as  to  dissociate  the  detection  rate  and  the  quality  of  the 
localization.  We  modified  the  module  to  allow  for  saving  previous  detection  results  and 
ground  truth  as  site  model  objects.  This  is  the  preferred  method  for  specifying  the  ground 
truth,  since  it  can  be  reloaded  and  is  more  easily  exchanged. 

Results  obtained  with  this  ground  truth  verifier  are  reported  for  the  Denver-TEC- 
2  images.  Figures  7  (a-c)  show  an  original  image  of  the  TEC-2  site  with  parking  lots 
delineated,  the  ground  truth  vehicles,  and  the  detection  results.  Table  3  shows  the  resulting 
statistics  for  the  TEC-2  images. 

Long- Road  Detector.  Most  IU  algorithms  developed  for  aerial  image  exploitation  fail 
to  take  into  consideration  situations  where  very  large  images  are  used.  These  images 
call  for  specific  computational  and  sometimes  algorithmic  solutions.  We  have  begun  to 
address  this  problem  in  the  context  of  vehicle  detection  on  long  roads.  To  this  end,  site 
model  information  in  the  form  of  the  road  model  is  fed  back  to  the  low-level  processing 
stages.  More  specifically,  the  Canny  edge  detection  module  is  modified  so  as  to  perform 
filtering,  gradient  computation,  non-maxima  suppression,  and  hysteresis  thresholding  only 
on  masked  regions.  The  block  image  structure  of  images  will  be  exploited  later,  with 
processing  performed  only  on  the  block  traversed  by  a  road,  which  further  limits  memory 
requirements.  Lastly,  template  specifications  in  the  case  of  a  long  road  will  consider  the 
facts  that  vehicles  are  often  isolated,  while  some  edges  in  parking  lots  are  occluded.  Results 
of  the  application  of  the  long-road  monitor  on  Ft.  Hood  images  are  shown  in  Figure  8; 
statistical  results  are  given  in  Table  4.  Note  that  on  the  Ft.  Hood  images,  the  detection 
module  is  applied  in  situations  where  it  is  known  to  perform  poorly,  i.e.  in  cases  where 
the  apparent  vehicle  size  is  below  7  pixels. 
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Main  modifications  and  Bug-Fix  History  . 

The  vehicle  detector  has  been  fully  integrated  into  the  RCDE  platform.  It  has  been 
implemented  in  C,  and  later  reimplemented  in  C++.  Its  current  integrated  version,  how¬ 
ever,  is  still  in  C.  The  early  timing  of  its  integration  and  testing  was  as  follows:  in  1993 
to  1994  the  basic  concept  was  tested  and  early  integration  occurred.  In  September  1994, 
the  2-D  version  was  integrated  into  the  RADIUS  testbed  and  tested.  In  Spring  1995,  the 
vehicle  detector  was  integrated  into  the  Lockheed-Martin  Valley  Forge  (VF)  Quick  Look 
(QL)  menus  and  a  3-D  version  was  delivered  to  GE  and  VF.  It  was  demonstrated  at  the 
San  Diego  ETS  demo  as  well  as  the  STISS  demo. 

The  integrated  vehicle  detector  has  been  tested  by  GE  and  Lockheed-Martin,  with  in¬ 
tegration  resulting  from  our  collaboration  with  these  institutions.  VF  has  provided  the 
needed  RCDE  upgrades,  the  testbed  version,  and  disk-saves.  It  has  provided  us  with 
feedback  with  bugs  and  fixes  as  well.  GE  has  provided  test  images  and  a  site  model  (Sch¬ 
enectady),  and  has  provided  assistance  in  getting  RCDE  upgrades  and  disk-saves.  It  has 
assisted  us  in  the  RCDE  integration,  particularly  with  the  QL  menus.  Over  multiple  and 
regular  integration  trips  made  by  R.  Welty  (GE),  GE  has  given  us  feedback  on  implemen¬ 
tation  issues,  bug  detection  and  fixes.  It  has  tested  compatibility  of  our  code  with  major 
RCDE  upgrades.  Some  examples  of  the  interactions  are  itemized  below: 

•  Formats: 

—  FBIP:  The  vehicle  detector  was  found  (VF)  not  to  work  on  FBIP  format.  It 
was  then  modified  (UMD)  to  work  on  FBIP  images  by  removing  all  references 
to  interior  camera  parameters  (Spring  1995). 

—  Lazy  Images 

*  The  vehicle  detector  did  not  initially  work  on  lazy  images  (sometimes  view¬ 
less)  (detected  at  VF,  May  1995). 

*  This  was  fixed  by  removing  references  to  views  (R.  Welty,  May  1995). 

•  Interactive  dimension  specifications: 

—  These  were  inadequate  in  their  initial  implementation  and  were  subsequently 
improved. 

•  Orientation  anomaly: 
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—  A  mis-alignment  between  image  coordinate  systems  and  2-D  coordinate  systems 
was  detected  (UMD,  June  1995). 

—  This  mis-alignment  resulted  in  great  deterioration  of  the  detection  performance. 
—  This  was  fixed  by  modifying  the  transformations  used  (UMD,  June  ‘95) 

Current  Features.  This  last  item  has  greatly  affected  the  detection  performance.  We 
expect  that  a  version  in  which  this  problem  has  been  fixed  will  be  included  in  the  next 
delivery  in  November  1995.  The  current  vehicle  detector  features  that  the  RADIUS  testbed 
should  have  in  the  next  delivery  will  include  the  following.  In  its  current  implementation 
the  vehicle  detector  is  accessible  either  stand-alone  or  under  the  QL  menus.  The  detector 
allows  for  dual  specification  of  the  vehicle  dimensions:  2-D  or  3-D.  It  enables  a  two-way 
interactive  specification  of  the  dimensions,  either  graphically  or  by  using  sliding  bars.  In  its 
current  implementation,  the  user  creates  a  sample  image  vehicle,  modifies  its  dimensions, 
then  uses  it  as  a  prototype.  The  user  can  modify  its  3-D  dimensions  directly.  The  sample 
vehicle  size  is  then  modified  accordingly.  Alternatively,  the  user  can  modify  the  sample 
vehicle  sizes  and  the  3-D  dimensions  are  modified  accordingly.  The  detector  accepts  FBIP 
format  and  lazy  images.  It  is  also  equipped  with  a  ground  truth  verifier. 

4  Conclusion 

We  have  presented  the  recent  work  performed  under  the  RADIUS  project  for  the  devel¬ 
opment  of  model-supported  algorithms  for  monitoring  vehicle-related  activities.  Special 
emphasis  has  been  placed  on  integrating  the  vehicle  detector  module  into  the  RCDE  envi¬ 
ronment. 
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Abbreviations  and  Acronyms 

The  following  abbreviations  and  acronyms  were  used  in  this  report: 


UMD: 

University  of  Maryland. 

VF: 

Valley  Forge  (Lockheed-Martin). 

GE: 

General  Electric. 

RADIUS: 

Research  and  Development  in  Image  Understanding  Systems. 

RCDE: 

RADIUS  Common  Development  Environment. 

FT: 

Fourier  Transform. 

GAD: 

Global  Activity  Detector. 

FBIP: 

Fast  Block  Interpolation  Projection. 

GSD: 

Ground  Surface  Dimension. 

QL  : 

Quick  Look. 
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Figure  1:  Canonical  Configuration 


Figure  2:  Typical  2D  Configurations 
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(a)  Areas  Monitored 


(b)  Active  Parking  Lots  for  MB2  Image  33 


(c)  Active  Parking  Lots  for  MB2  Image  23  (d)  Active  Roads  for  MB2  Image  18 


Figure  3:  Global  Detector:  Monitoring  of  Parking  Lot  Occupancy  and  Road  Convoys. 
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(a)  Active  Roads  for  MB2  Image  20  (b)  Active  Areas  for  MB2  Image  10 


(c)  Active  Areas  for  MB2  Image  19  (d)  Active  Areas  for  MB2  Image  33 

Figure  4:  Global  Detector:  Monitoring  of  Parking  Lot  Occupancy  and  Road  Convoys. 
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(a)  Backprojected  Locations  (MB2  Image  10)  (b)  Backprojected  Locations  (Enlarged) 


(c)  Detected  Vehicles  (d)  Detected  Vehicles  (Enlarged) 


Figure  5:  Backprojection  and  Local  Vehicle  Detection 
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(a)  Parking  Lot  6.  (b)  Detected  Vehicles  on  Parking  Lot  6. 


(c)  Parking  Lot  9.  (d)  Detected  Vehicles  on  Parking  Lot  9. 


Figure  6:  Vehicle  Detection  Results  on  Ft.  Hood  Image  4. 
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(a)  Detected  Vehicles. 


(b)  Original  Image.  (c)  Ground  Truth  Vehicles. 


Figure  7:  Vehicle  Detection  Results  on  Denver  Image. 
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Ft.  Hood  Image  2  (Detail). 


Ft.  Hood  Image  2  (Detail). 


Figure  8:  Long  Road  Monitoring  on  Ft.  Hood  Image  2. 
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Table  1:  Detection  Results  on  Roads  and  Parking  Areas 


Areas 

Active 

Inactive 

Pf(d) 

W) 

Parking  areas  (learned/optimal) 

69 

86 

0.1123 

0.9714 

Parking  areas  (hand-tuned) 

69 

86 

mm 

Road  segments  (hand-tuned) 

24 

0.8340 

Table  2:  Detection  Statistics  for  Ft.  Hood 


image-parking  lot 

true 

veh. 

correct 

det. 

false 

det. 

missed 

det. 

det. 

rate 

false 

alarms 

4-6 

47 

41 

3 

6 

87.23  % 

6.81  % 

4-8 

166 

150 

13 

16 

90.36  % 

7.97  % 

4-9 

218 

201 

24 

17 

92.20  % 

10.66  % 

4-11 

224 

187 

14 

37 

83.48  % 

6.96  % 

4-al 

33 

22 

1 

11 

66.66  % 

4.34  % 

4-a2 

40  [ 

31 

1 

9 

77.50  % 

3.12  % 

4-a3 

74 

58 

5 

16 

78.37  % 

7.93  % 

4-a4 

91 

69 

1 

22 

75.82  % 

1.42  % 
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Table  3:  Detection  Statistics  for  TEC-2 


image-parking  lot 

true 

veh. 

correct 

det. 

false 

det. 

missed 

det. 

det. 

rate 

false 

alarms 

34-1 

79 

67 

15 

12 

84.81  % 

18.29  % 

34-2 

53 

36 

4 

17 

67.92  % 

mm 

34-3 

51 

45 

9 

6 

88.23  % 

16.66  % 

34-4 

149 

118 

12 

31 

79.19  % 

9.23  % 

34-5 

165 

122 

10 

43 

73.93  % 

7.57  % 

34-6 

165 

150 

26 

15 

90.90  % 

14.77  % 

Table  4:  Detection  Statistics  for  Ft.  Hood  Long  Road 


image 

correct 

det. 

false 

det. 

missed 

det. 

det. 

rate 

false 

alarms 

ma 

8 

0 

6 

■iMIBba 

0.00% 

Ft.  Hood  image  3 

■a 

23 

0 

11 

mm 

0.00% 

MB2 

50 

44 

4 

6 

88.00% 

8.33  % 
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